Accountability in evaluation – accountable to who?
Written by Patrick Regan
Being accountable, and being held responsible for the results of our projects and activities is one of the key pillars that frame monitoring and evaluation practice. The growing and varied acronyms for evaluation professionals sometimes include an ‘A for Accountability, e.g. a MEAL manager. Before you get hungry, MEAL stands for Monitoring, Evaluation, Accountability and Learning.
As a consultant focusing on the evaluation of human rights programmes, the concept of accountability is harmonious with the lexicon of holding governments and state actors accountable for their human rights violations. Promoting accountability is surely a good thing, and having a dedicated role to focus on accountability sounds like a good thing to have at an organisation…right? Although conceptually this makes sense, this brief article highlights some of the risks incurred by organisations when accountability processes and structures are not carefully thought through.
Organisations should be asking themselves, “Who are we holding ourselves accountable to?”. Many organisations will prioritise accountability to their donors, and this is where accountability could end up doing more harm than good – especially if you then structure monitoring, evaluation and learning plans which prioritise donor accountability. With a donor-first approach, you risk ending up with problematic indicators of success and diverting your attention away from what meaningful impact looks like for the people you are trying to support and the systems you are trying to influence. This approach can then have a knock-on effect on the perceived value of monitoring and evaluation across the organisation – project delivery staff view the process as donor box-ticking instead of an opportunity to learn, improve and maximise the impact of their work. This in turn makes it harder to engage staff and stakeholders in the evaluation process, potentially limiting the quality (and quantity) of the data you have. In the end, a donor centred approach to accountability and MEL is unlikely to succeed.
This situation can be exacerbated for human rights organisations where the operating environments can be incredibly complex, change can be hard to observe and the impact can, at times, be more abstract, resulting in an approach to learning and accountability made up of vanity metrics and meaningless indicators which do not provide any real opportunity to learn or gain insight.
We should also be looking at accountability from the perspective of structural power imbalances. Most human rights funding comes from wealthier governments, family trusts and foundations, and individuals philanthropists, i.e. groups who already have a significant influence in the world. Therefore, is it ethical for us to prioritise accountability to them – does doing this perpetuate further inequalities? And do donors even want organisations to do this?
So who should we be holding ourselves accountable to?
This question can lead to a lot of (potentially useful) debate within organisations. If we are working in the public interest and seeking to advance the rights, protections and lives others, should we not be prioritising accountability to those we claim to be acting in the interests of?
I would encourage organisations that are serious about improving their accountability, and looking to build a stronger culture of monitoring, evaluation and learning within their organisations to focus on holding themselves accountable to the groups, communities, individuals and organisations they are seeking to support. Rather than asking yourselves, “Did we deliver the activities and results promised to our donors?” start by asking:
What will results and impact look like for those we are claiming to act in the interests of?
What would realistic and meaningful outcomes be for these stakeholders? (and how much variation is there between these groups and individuals)?
From the perspectives of these different groups, what would be a meaningful indicator or progress marker that these results are materialising?
Are there ways we can involve these groups, or representatives from these groups, in the collection, analysis or interpretation of evaluation data?
Developing a monitoring, evaluation and accountability framework from this starting point could make the entire process more meaningful, learning oriented and potentially promote more significant results.
Organisations should also be motivated to hold themselves accountable to themselves. Organisations might want to consider:
What information do we need as an organisation to know that we are pushing things in the right direction?
How will we know things are going to plan?
Are we satisfied with how effective and efficient our activities are?
How are we responding to challenges and changes?
What is the unique role of our activities and programmes in the wider eco-system and how important is our contribution?
Holding ourselves accountable to those we are seeking to support, and being accountable to ourselves are likely to be harmonious. If accountability is happening at these two levels, and your MEL systems are structured accordingly, you should be well-positioned to propose a MEL approach to your donors that holds you accountable to them in the same way – meaning donor accountably does not become a driving force but is second nature to your own accountability practices. In my personal experience, the majority of donors I have worked with are adaptable, flexible and willing to support your own MEL structures and strategies if they can see time and thought has gone into doing these things appropriately, and they can see they are working.
Taking action
In summary, I advocate for and encourage organisations to prioritise holding yourselves accountable to the groups whose interests you claim to represent, closely followed by internal organisational accountability – s simple recommendation, which is more complex to realise than it is to suggest. But putting the time into this process could help you to shift the axis of power, generate more useful and meaningful insights and learnings which flow effortlessly into your donor accountability structures.
Some simple first steps in this process could be:
Conduct a light touch accountability assessment, mapping your organisation’s accountability structures and processes, identifying the strengths and weaknesses of your current approach, and understanding who or what is being prioritised and why.
Engage with individuals and groups that you are supporting and who are the target of your projects and activities – find out what success would mean to them, and what results would be meaningful for them? Consider if your conceptions of “impact” are coherent with theirs.
Consider the extent to which you are involving these individuals and groups in the design of data collection tools, the collection of data, analysis, interpretation and sharing of findings. Is there room for greater engagement and involvement in these processes?
Consider what mechanisms you have in place within your organisation to hold yourselves accountable at a result or outcome level. How are you monitoring your outcomes? and who is involved in using, interpreting, and responding to outcome monitoring information and data?
Consider who is responsible for MEAL in your organisation – if you don’t have a dedicated individual, do these tasks end up with your fundraising team? If so, how is this affecting your organisation’s approach to accountability?
Find out what data, information and results are most interesting and useful to your donors, and how much flexibility there is to explore different approaches or to reframe and reorganise your indicators and milestones to prioritise accountability to target groups – you might be surprised with their flexibility and openness to experiment.
The accountability debate of course has many layers and complexities, and these are just a few thoughts and reflections based on my personal and professional experiences of accountability and evaluation.
Rights Evaluation Studio provides a range of services including project design, strategy, monitoring, evaluation and impact assessment. Please get in touch if you would like to discuss how we can help you to review, update or develop monitoring and evaluation systems that work for your organisation.